DOCUMENT RESUME ^> 

; SE 040 196 

Herbert , Martin 

Student Achievement^ Rapid Implementation Model . 
^Evaluation Re^port Extended Pilot Trial of the 

Comprehensive School Mathematics Program* 
CEMREL, Inc., St. , Louis, Mo. 

National Inst, of Education (ED)>^ Washington, DC. 
Dec 80 

35p^*; For related documents, see SE 040 1^^195 and 
SE 040 340-348. 

Reports - Research/Technical (143)* 
MF01/PC02 Plus Postage. 

Educational Research? Elementary "^Education; 
*Elementary School Mathematics; *Math^atics 
Achievement; Mathematics Curriculum; Mathematics 
Instruction ; *Program Evaluati on; *Test ing 
*Comprehensive School Mathematics Program; 
*Mathematics Education Research 

The Comprehensive School Mathematics Program (CSMP) 
is a program of CEMREL, Injc., one of the national educational 
laboratories, and was funded by the National Institute of Education 
(NIE). l\s major purpose is ^he development of curriculum materials 
for grades kindergarten through 6. Twq schools began using CSMP in 
fall 1978, with the curriculum begun with all stud^ts through fourth 
grade, rather than the more common , grade-by-grade approach that 
begins with, only first-grade classes. A series of 'tests administered 
ih 1978, 1979, and 1980 were designed to compare th^, pe rf otmance of 
students before the introduction of CSMP, .after one*^year's experience 
and after two years' experience. Results indicated: ^i) On MAN'S tests, 
significant improvement was found from 1978 to 1980,>which was 
^ consistent with previous CSMP and non-CSMP comparisoas made with the 
same tests; 2) The kinds of tests which the 1980 group did relatively 
better on were also ones on which CSMP superiority had been 
demonstrated in previous studies; and 3) On second-grade standardized 
' tests, virtually no change at either school in mathematics score.s 
related to reading scores. However, there was usually small 
decrease in the first year of CSMP for third- and fourth-grade 
pupils, followed by a "more-than-compensati ng" gain the^ second year^^ 
^ (MP) ^ • ^ * 



ED 225 837 

AUTHOR 
TITLE 



INSTITUTION 
SPONS AGENCfY 
PUB DATE 
NOTE 

PUB TYPE , 

EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



ABSTRACT 



**************************************************************** 

* Reproductions supplied by EDRS are the best that can be made * 

* from the original document. * 
*********************************************************************** 

\ 



9^- ^ 



imPKBHGpiVG MHIMIL HEAIA i ItSi IMMIOItAAl: 
BVAMIATION KEIHIKT SKKIGS 

P ; - Student Achievement, Rapid Implementation Model 

us DEPARTMENT OF EDUCATION 

^ * NATIONAL INSTITUTE OF EDUCATION PERMISSION TO REPRODUCE THIS 

t^Du-A^'0\A, "bS(A.H.^s -.MiKMATiON MATERIAL HAS-BEEN GRANTED BY 



TO THE EDUCATIONAL RES0UR6eS 
INFORMATION CENTER (ERIC) ' 




Extended Pilot Trial .of the 
Comprehensive School Mathematics Program 



Evaluation Report 7-B-6 
Student Achievement, Rapid Implementa^tion Model 

I 

f 

i 

Martin Herbert 
December, 1980 



Developed by CEMREL, Inc., a priv^ite nonprofit" corporation supported 
in part as an educational .laboratory by funds from the National 
Institute of Education, Department of Health, Education, and Welfare. • 
The opinions expressed in this publication do not necessarily reflect 
the position or policy of the National Institute of Education, and 
no official endorsement should be inferred. 

Copyright on these materials is claimed only during the period of 
development, test, and ^valuation, unless additional authorization 
is* granted by the National Institute of Education, to claim copyright * 
on the final materials. For information on the status of the copyright 
claim,- contact either the copyright proprietor or the National Institute 
of Education. 



ii 



•i 



ERIC 



Description of Evaluation Report Series 



The Comprehensive School Mathematics Program (CSMP) is a program of CEMREL, 
Inc-, one of the national educational laboratories, and is funded by the National 
Institute of Education/ Its majar purpose is the development of curriculum 
materials for grades K-6.' - * 

• « 

Beginning in September, 1973, CSMP be^an an extended pilot trial of ^*ts 
Elementary Program. The pilot trial is longitudinal in nature; students who 
began using CSMP materials i^i kindergarten or first grade in 1973-74, were able 
to use them in first and second grades respectively in 1974-75, and so on in 
subsequent years. Hence the adjective "extended". 

The evaluation of the program in this extended pilot trial is intended to be 
reasonably comprehensive and to supply .inf6rmation desired by a wide variety, of 
audiences,. For that reason the reports in this series are reasonably non-technical 
and do not attempt to widely explore sofne of the related .issues.^ The list of reports 
^ through yea^r six is given on the next page. The following reports are planned for 
year 7: 

7-B-l - Fifth Grade Evaluation: Volume h Summary 

7-B-2 - Fif^ Grade Evaluation: Volume II, Test Data 

7-B-3 - Fifth Grade Evaluation: Volume III, Non-Test Data / - 

7-B-4 - Re-evaluation of Second Grade;, Revised MANS Tests 

7-B-5 Achievement of Former CSMP Students at Fourth GrAde 

7-B-6 - Student ^Achievement, Rapid Implementation Model 

) 



\ 



ill- 



lERlC 



Extendefl Pilot Trials of the 
Comprehensive School Mathematics Program 



Evaluation 
Evaluation 
Evaluation 
Evaluation 
Evaluation 
Evaluatiop. 
Evaluation 
Ev£iluation 
tvaluatipn 
Evaluation 
Evaluation 
Eva luation 
Evaluation 
Evaluation 
Evaluation 

Evaluation 
Evaluation 
Evaluation 
Evaluation 
Evaluation 
Evaluation 
Evaluation* 



Report 1-A-l 
Report l-A-2 
Report l-A-3 
Report^ 1-B-l 
Report l-B-2 
Report l-B-3 
Report l-B-4 
Report l-B-5^ 
Rept^f4;^l-B-6 
Report 1-C-l 
Report l-C-2 
Report l-C-3 
Report l-C-4 
Report l-C-5 
Repoi^t l-C-6 

Report 2-A-l 
Report 2-B-l 
Report 2-B-2 
Report 2-B-3 
Report 2-C-l 
Report 2-C-2 
Report 2-C-3 



1 



Evaluation Report' 3-B-l 
Evaluation Report 3-C-i 

Evaluation J^epo^t 4-A-l 
Evaluation Report 4~B-1 
Evaluation Report 4-B-2 
Evaluation Report 4-B-3 
Evaluation Report 4-C-l' 

Evaluation Report 5-B-l 
Evaluation Repart 5-B-2 
Evaluation Report 5-C-l 



Evaluation Report Series 

Overview, Design and Instrumentation 
- External Review of CSMP Materia]j('s 
Final Summary Report Year 1 
Mid-Year Test Data: CSMP^ First Grade Content 
End-of~Year Test Data: CSMP First Grade Content 
End-of-Year Test Data: Standard First Grade Content 
End-of~Year Test Data: CSMP Kindergarten .Content 
Test Data on Some General Cognitive Skills 
Summary Test Data: Detroit Schools 
Teacher Training Report 
Observations of CSMP First Grade Classes 
Mi4;-Year Data from Teacher Questionnaires 
End-of-Year Data from Teacher Questionnaires 
Interviews with CSMP Kindergarten Teachers 
Analysis of Teacher Logs 
/ 

Final Summary Report Year 2 
Second Grade Test Data 

Readministration of First Grade Test Items 
Student Interviews 
Teacher Questionnaire Data 
Teacher Interviews, Second Grade 
Teacher Interviews, First Grade 

Second and Third Grade Test Data Year 3 
Teacher Questionnaire Data Year 3 

Final Summary Report 'Year 4 ^; 
Standardized Test Data,, Third Grade 

Mathematics Applied to 'Novel Situations (MANS) Test Data 
Individually Administered Problems, Third Grade 

Teacher Questionnaire Data, Third Grade I 

^ >- 

Fourth Grade MANS ,Te6t Data 

Individually. Administered Problemsi Fourth Grade 
Teacher Questionnaire and Interview Data, Fourth Grade 



Evaluation Report 6-B-l 
Ev'aluation Report 6-B-2 
Evaluation Report 6-C-l 



Comparative Test Data: 
Preliminary Test Data: 



Fourth Grade 
Fifth Grade ' 

Grades 3-5 



Teacher Questionnaire Da^a; 

^ ^ - Key to Indexing 

Evaluation Reports are labelled m-X-n, 

i 

where m is the year of the pilot study, with 1973-74 as Year I*^- ^ 
X is the .type of data being reported where A is for overviews 
and summaries, B is for student outcomes and C is for other data* 
[ n is the number within a given year and type of data* 



Table of Contents 



Summary ^ 1 

Setting 3 

Second Grade Results 5 

Tlrird Grade Results 9 

Fo'urtn Grade Results 13 

Appendix A 17 

Appendix ^21 

Appendix C . . . ^ 27 



Summary 



Two schools, one in a large Southern city and the other in a medium 
sized Midwestern suburb, began using The Comprehensive School Mathematics 
Program (CSMP) in fall of 1978. At that time CSMP was begun with all 
students through fourth grade rather than the more comnon grade-by-grade *v 
approach beginning only in first grade. 

A series of tests was administered in the spring of 1978, 197S and' 
1980. The purpose was to compare, at a given grade level, the performance 
of students: before the introduction of C^MP (1S78), after a year's 
experience with CSMP (1979), and after two years' experience with CSMP (1980). 
The tests were administered in grades 2-4. 

The fijain results were the following: 

• On the MANS tests, a series of tests designed to assess some of the 

^ underlying processes of CSMP without using any of the special terminology 
or problem situations of the curriculum, there was from 1978 to 1980 
significant improvement at every grade level and this improvement was 
very consistent with CSMP-non-CSMP comparisons made previously with the 
same tests. 

• At second grade this improvement took place in the first year, from 1978 
to 1979, with no further change from 1979 to 1980, At third and fourth 
grades, there were modest gains the first year and further gains the 

• second year. 

• The kinds of tests on which the 1980 students did relatively best - number 
relationships, mental arithmetic and estimatfon - were also the ones for 
which there had been a demonstrated CSMP superiority in previous studies. 

• On standardized tests, at second grade, there was virtually no change at 
either school in^ math scores relatiye to reading scores. At third and 
fourth grades there was usually a small decrease iji the first year of 
CSMP followed by/a more-than-comp.ensating gain the! second year; scores 
on the Concepts tests always increased relative to Computation^ scores. 
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Setting 



Concurrent Impl ementation 



The CoT^^prehensi ve School Mathematics Program (CSMP) is a K-6 math currficulum 
developed with the expectation that it would be introduced in schools on a 
sequenti'til basis, one grade at a time. This has usually been the case, but 
occasional ly -school s have decided to begin the program in several grades 
concurrently, the intent being to compress the implementation effort into a 
single year of concentrated effort. The present study was an attempt to 
investigate, student achievement in two schools which opted for this "concurrent 
implementation" model. 

i ^ 

Participating Schools ; 

In.the'fall of 1978, twO;Schools began using CSMP, 'for the first time, as 
their math curriculum for all.students, grades K-4, - Brief descriptions of these 
tvyo schools are given below. 

School A: - an inner city school in a large southern city 

I - a large school,- g^fades K-6 with T50-200 students per^rade level 

- student and teacfjer population virtually all black 

- rather low student achievement; mean percentile ranks of 
35-40 on ■ standardized achievement tests 

* ? 

- Osed two resourle teachers, paid for by a state grant, to train 
teachers in thelregular classrooms over the course of the year 

School B: - a school in an |older suburb of a large midwestern city 

- a medium sized/ school , grades K-5,'with 70-90 students per grade level 

-'student poputcftion mostly white, though the distrrct is more than . 
50%;bl^ck j- ^ 

- gen'erally vqfry high student achi.vement; mean percentile ranks of 
^ 75-/85 on st|hdardized achievement tests 

First gra^^ers in bqlh schools used the regular first grade CSMP, program. 
Second and thi.t^d grader^ began with a* series of special introductory lessons, 
then moved fdilrly quickly into the regular programs. Fourth graders spent the 
first semester on the fourth grade entr^ module, which was developed especially 
for classes pf studentsf beginning the frogram in fourth grade; consequently these 
students dic|Uot begin/the regular fir$t semester's work until the second 
semester and would remain a semeste^ behind^in later grades, 

J 



■/ , • . 

Testing Plan ^ 

A series of tests was administered on three occasions to all students^ 
in grades 2-4: 

* * spring, 1978 ~ The students had no CSMP experience. 

spring*, 1 979 - The^tudents were completing their first year gf CSMP. 

spring, 1980 - The students were completing their second year of C^P. 

It was then possible to compare the achievement of students at a given grade 
level. For example, the achievement of second graders in 1978 (no CSMP experience) 
could be compared with that of the next year's second graders in 1979 (one year of 
CSMP) and with the achievement of the following year's second graders in 1980 
(two year's experience). 

Three kinds of tests were used: 

a) The math subtests of whatever standan^ized tests were routinely used 
by the district. This data was provided by the respective school 
districts. 

b) The MANS tests, a series of short te^t scales designed for previous 
evaluation studies by CEMREL's Mathematics Research and Evaluation 
Studies program.^ Most of these tests were inte^rded to assess some 
of the underlying thinking skills of CSMP without using any of the 
terminology or problem situations of th$ program. They were 
administered by special ly^ trained testers. 



c) The reading comprehension test of standardized test used by the 
distrixt. The data from these tests served as an indicator of 
whether or not the various groups of students to be compared wel,re 
of similar ability levels; in addition these scores could be us.ed 
as covariates in- the analysis of the other test data. ^ 



^ Third graders in School B were not tested in 1980^ 

^ See Evaluation Report? 3-B-l , 4-B-2 and 5-B-l for descriptions of tests and 
test results for the original administratjion of these tests- 
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Second Grade Results 

MANS Tests ' \ ' . 

For each class, a mean raw score was calculated for each MANS Test. 
Brief descriptions and s^jmmary data for these individual scales are given in 
Appendix A. The Total MANS score for each class was also calculated and plotted 
against the corresponding mean score in reading. Different readir^ tests were 
used in the two schools; furthermore School B changed standardized tests after 
.the ^first year' Hence, all reading scores were converted to percentile ranks 
for the graph in Figure 1, below. 
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Figure 1, Graph of Cla^ss Means for Second Grade Total MANS 
• = 1978 class, x = 1979 class, 0= 1980 class 

It can be seen from the regression line which has been drawn on the graph 
that there is a trend for the 1979 and 1980 classes to have higher §cores 
relative to their ability Tevel (i.e. above the regression line for the 1978 
classes). There appears to be no difference between the 1979 and 1980 classes. 
Thus with the introduction of CSMP in 1979 there was an immediate improvement, 
but the 1980 group, which had the benefit of studying CSMP in first grade^as 
well, did not make further improvement. 



This resuTt can be Illustrated in a different way. For each school the 
mean score across classes each year was calculated and graphed together with 
mean scores from other districts who had participated in the original evaluation 
with these. tests. These other districts had adopted CSMP in the usu-al year-by-year 
v?ay, and for them, the comparison was with similar non-CSMP classes in the * 
district. This graph is shown in Figure 2, below; with the' common regression 
line. 
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Figure 2, Second Grade MANS Scores, Present StudyPlus Previous^Data 
AO, Al, A2 = school A in 1978, 1979 and 1980 respectively 
School B in 1978, 1979 and 1980 respectively 
CSMP District (previous study) 



BO, 81, B2 

X 



non-CStIP Distr.ict (previous study) 



Figure 2 shows that the data from the present study agree with the previous 
dal:a rather well (based on^the comnon regression line) if 'one simply considers^ 
the 1978 testing as "Non-CSMP" and the 1979 and 1 980^estings as "CSf^P". It 
also shows again that there is virtually no difference in the scores from 1979 . . 
and 1980. 
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Thus the pattern of results in these two schools was remarkalxly similar ' 
to results obtained -previously in which there was a significant advantage for 
CSMP classes. Significance* tests were not calculated in the same way in the 
present studies because of the small number of classes an^different reading 
measures. When analysis of covariance was used wi*h student as the unit of 
analysis - perhaps an fnappropriately liberal method - g^'ns on the total 
and on almost all the individual scales were significant at both schools from 
1978 to 1979, but not from 1979 to 1980. 

-However, although the pattern of tota] score gains was similar^ there were 
some differences with respect, to individual scales. Two scales dealing with 
the production of multiple ansVvers, ar fluency (A2, Equation Fluency and Bl 
Number Sentences), produced the largest gSiins but neither was significant in 
the original study. On the other hand, two scales dealing with the easy 
computation of large numbers produced small and irregular gains in the 
present study compared with large and^ §ignif icant differences in the 
'original study. ^ Appen'Six A gives tj||| school means for each scale use^? in - 
the second grade. 

Standardized, Tpst Data 

At School A, the Comprehensive Test of Basic Skills battery was administered 
each year as part of the regular district testing. 

A meaa score for each class was Calculated for the math subtests and for the 
total reading test and converted to a percentile rank. The mean across classes 
for each of the various tests is. given in Table 1, belpw. 

Table \ 

. Mean Percentile Ranks* School A 
-Second Grade Comprehensive Tests of Basic Skills 



Year 


Reading 


Math 
Computation 


Math 
Concepts 


Total 
Math 


1978 


35.0 . 


58.0 


35.2. 


46.4 


1979 


40.2 


55.4 


44.8 


51.0 


1980 


38.2 


52.0 


44.5 


48.3 



For all years, the percentile for Total Math is about 10 points more than 
for Reading. Over. the course of the two years there has been a small-. drop in 
Computation scores and an increase in Concepts scores. 
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At School 'B, the Stanford Achievement Test was administered in 1978, , but 
the California Achievement Test was used in the district in'lQyg and 1980. 
Thus, norm data frogi^'these two tests are not directly comparable. Table 2 
presents the mean scores on these tests. 



Table 2 

Mean PerC^entile Ranks, School B 
Second Grade Standardized Tests 



Year of Testing 


Total 
Reading 


Computation 


Concepts 


1578 (Stanford Achievement Test) 


78.5 


72.7 


82.3 


-1979 (California Achievement Test) 


81 .3 


79.4 


80.6 


1980 (California Achievement Test) 


85.0 


81.6 


88.0 



Because of the change in tests, and because of the very high scores (with 
potential ceiling effect) on all tests, it is difficult to evaluate any changes 
1n ithe data. ^Computation scores remain slightly lower and Concepts slightly 
higher than the corresponding Reading scores. 
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Third Grade Results '. 

MANS Tests 

Figure 3, below, shows the graphs of mean class scores on the Total of 
the MANS scales versus percentile rank in reading. It should be noted that 
these tests were not administered in 1980 at School B; hence there are no(3)fs 
in the tapper right corner, of the graph. 
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Figure 3, Graph of Class Means for Third Grade Total MANS 
• = 1978 class, x = 1979 class, ®= 1980 class 
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A£ both sites there has been an improvement in scores from year-to-year. 
At school A this effect is lessened somewhat in 1980 by one low scoring class: 
particularly low relative to its reading ability. Furthermore, there is no 
regression line that adequately fits the data at both schools, 

9 



In Figure 4, the school means are graphed together\w1th means derived from 
previous studies, and a common regression line. 
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Figure 4, Third Grade MANS Score^, Present Study Plus Previous Data 
AO, Al, A2 = School A in 1978, 1979 and 1980 respectively 
BO, Bl, B2 = School B in 1978, 1979 and 1980 respectively 

X = CSMP district, • = non-CSMP district - previous study 

School A fits the previous data fairly well; a small g^ii^n^after a year of 
CSMP (from AO 'to AT) and a further gain the second year (aT to A2). Unlike the 
second grade where there was a. healthy gain in the first year and none after 
another year, it appears that by the end of third grade, students who started 
the program at third grade do better than non-CSMP thi rd graders, but less wetll 
than third graders who started the program a year earlier in second grade. 
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At Site B there was a large gain (larger than'at any either site) after just 
a year's use of CSMP. * 



"■"he pattern of gains on indi v*idudl scales^in Schools A and B was simvlar tc 
the pattern in the original study. On the four scales in which significant 
differences were* not^ previously found, there wei^e again very sma>l differences 
except for on^ scale at one schocJ'l On the remaining 10 test.s- on whjch there 
were ''again .correspond irjg gains for eight of the ten at each of the two schools 
Appendix B givis the school means for each scale used in the third grade. 



Standardized Test Data ' ^ . _ 

• f 

For School A, the percentile ranks corresponding to the mean across' 
classes on the -Compr'ehensi ve Tests of Basic Skills are shown in Table. 3, 
below. I ' * 

Table 3 - ' ' 

Mean Percentile Ranks-, School A ' " r 
Third Grade Comprehensive Tests of Basic ^ills 



^ Reading 
Year Conlprehension 

1 978 44 
1 979 56 
1 980 57 



Math 
Computation 



55 
63 
67 



Math 
Concepts 



37 
54 
57 



Math 



Appl iciations'> Jotal 



36 
38 
44 



Math 



45 
58 
64 



It can be seen that Total Math scores improved slightly more than 
Reading scores between 1978 and 1980, and that, of the indifvidual math 

> 

subtests, the Concepts test showed the largest gain. 

Table 4 shows mean percentile ranks on the standardized test data from 

School B. Data was not collected in 1 980. * 

<. ' * ' If 

Table 4 - , ^ 

Mean Percentile Ranks, School B' 
Third Grade Standardized Test Data • ^ 



r — ^ 

Year and Name of Test 


Reading 


Math 
Computation 


Math 
Concepts 


Math . 
App.l i cat ions 


Math 
Total 


1 978 (CTBS) 
1^79 (CAT) 


88.0 
83.1 


88.5 
79.9 


* 88.2 
86.8 


86.5 


89.4. 
84.8 



The decline in Reading and Total Math scores (possibly a result of the 
cl^ange in tests and their respective norms) was very similar; relative to 
Reading scores, there was a decrease in Computation scores and an increase in 
Concepts scores. 
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Fourth Grade Data 



MANS Tests 



At School A, it was necessary to administer the third grade MANS tests^ 
to the fourth grader^ (the fourth grade scales Viot being available by the 
testing date at School A). ^ 



The mean scores on the totals of these MANS Tests were: 



1978: 48.8, 



1979: 60.9, 



1980: 64.9 



When the scores ar% adjusted for differences in reading ability from 

if 

year-to-year, the^scores go from 48 to 59 to 67, i.e. large gains in each of 
the two y€ars. 



Figure 5 shows the class means for each of the three years on the total 
of the MANS scales. Relative to the regression line 'tor the 1978 classes, 
scores are higher in 1979 and again in 1980. 



\ 
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Figlire 5, Mean Scores on Third Grade MANS Scores 
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I, x,0= 1978, 1979, 1980 Fourth Grade Class Means, School A 
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At Schdol Br on the regular fourth grade MANS tests, the me^ scores 
increased fnqm h?374 to 141.7. to >50.4. These mean scores are plotted in 
Figure 6, bellow, along with various class means from the<previous study. 



Total MANS 




Figurle, Fourth^rade MANS Scores, School B 
BO, Bl, B2 = Sc%pl B m4ns for 1S(78,' 197^, 1980 respectively 



X = tSMP CI 



• = Non-CSMP Class - previous study 
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^Reading or Ability 
^ Scor€ 



The pattern of gains at Scf%l B from 1978 to 1980 was very similar. to 
the pattern of CSMP-non-CSMP difm^ recorded in the earlier study. For 
17 scales there vJere significant oi^early significant differences on each 
occasion and on 5 scales there were "mrtually no differences on. either occasion, 
School B in 1980 did relatively better%han previous CSMP classes on a 
computation scale with division and on ^estimation scale which also used 
a division process. 



'Standardized Test, Data 

Mean Standard Scores across classes at School A are given in Table 5 
below. 



Table 5 

• r 

- Mean Percentile Ranks, School A 
' Fourth Grade Comprehensive TesJ:s of Basic SkilVs 



Year . 


Reading 
Comprehension 


Math 
Computat'i on 


Math 
Concepts 


Math 
Applications. 


Math 
TotaJ 


1978 


' 405 


391 


391 


\370 


386 


1979 


407 ' 


389 


383 


. ^362 \ 


376 


1980 


- 387 


390 


410 


379 j 


387 



Fi^om 1978 to 1979, with Vittle change in Readfng scores, there was a small 
decline in each of the Math scores- But in 1980, with a large drop in Reading ' 
scores, there were gains in Concepts and in ApplicaHqns. If scores were .adjusted 
for Reading ability, these gains would have been equival^eQt to about 10-15 
percentile ranks from 1 978. \. 

Standardized test dlata from fourth grade. School B, is summari^Bd in 
Table 6, below. I X 



. Table 6 
Mean. Percentile Ranks, School B 
Fourth Grade, Standardized Tests 



Year (Test) 


Total 
Reading 


Math 
Computation 


Math 
Concepts 


Math 
To£al 


1978 (CTBS) 

1979 (CAT) 

1980 (CAT) 


79.0 
82.2 
86.9 


78.7 
74.9 ^ 
83.2 


78.9 ' 

81.1 

89.1 


79.4 
79.3 
88.4 



At School B, there were very small changes in Math scores relative to 
Reading, though there, was again a tendency for scores to drop the 'first year 
of CSMP (1979) and to increase the next year. Again, after CSMP, Computation 
scores tended to be lower and Concepts scores higher than Reading scores. 
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Appendix A 

^Second Grade MANS Scales and /School Means 



1 



^ The means for each scale are given for Schools A and B for 1978, 1979 and 
1980 on page 19. 



MANS A ' . • 

Subtest 1: Sequences The crucial directions were, 'The nunfcer^ are 
in a certaiji o^der. . .tigure out what the pattei?i is... put the right 
nunber in the box.'* 



Exanple: 



16, 13, 10, 



Subtest 2: Eqi^tion Fluency The crucial directions were, "...write 
as many-true nimber sentences as you can, using only these signs and^ 
nunfcers." « ♦ - x 1 2 3 

Exmple: 3-1 = 2 

Subtest 3: Fuictions The crucial directions were, ''A teacher was 
playing a gaine with the class... a student gave her 2 (the first 
nunber of each pair), she did something to it and g^t 4 (the secaid 
nirber) ...figure out what if was that the teacher was doing to the 
nunbers, and then p«t the right nunber in the empty box." 



Exanple: 



10 



14 



10 



Subtest 4: Nuyi)er Line Estijuaticn The crucial directions were, 'This 
Is a funny looking nunber line isn't it? there are a whole lot of 
problems (13) and you won't have much timeH24 min.) to do them. You 
should not try to calculate the exact answer; just^cide quickly where 
the answer would probably go on the nunber liner** 

29+29 



Exanple: 



0 



r ■ 

4- 



10 



-f- 

50 



100 



500 



1000 



SiAtest 5: Confutation The crucial directions were, "...figure out 
what goes in the boxes to make the nunfcer sentences true." 



Exaiople: 



□ 



49 - 0 



See CSMP Evaluation Report 4-B-2 for a fuller description of these tests and 
the results from the CSMP-non- CSMP comparison involving 55 classes. 
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MANS B 



Subtest 1: Labelling Nunfcer Lines The crucial directions were, 
"...jfigure out what nuiTber would go in the box on each number line>'* 
(Note to reader: no student did both this subtest and Subtest 4 of 
Booklet A in which the nunfcer line was not equal -interval.) 

Exanple: ^ h + 1—4 ^ ^ :H h> 

20 25 30 ' ^ 45 50 55 60 



Subtest 2: Number Sentences The crucial directions were, 'Vrite 
nunfcer sentences for 8... write as jnany as you cw think of." 

Exajiple: 8 « 6 2 ^ 

Subtest 3: Word Problems The crucial directions were, 'Tor each 
series of pictures there is a story. At the end of each story there 
is a question you will be expected to answer/' 

4 

Exajnple: "First picture, ^Four children each get the sarc 
allowance from their mother • ' Second f)icture, 
'The four childrentput their allowances together/ 
Third picture, 'They have altogether 12 dollars.' 
Question, 'HowWch did each child get?"* 

(Note to reader: while the above information was being read to the 
students, the student's test page ccntained the pictures below.) 




Hffw vuch did each child get? 



Subtest 4: Nunfcer Sentence Pictures This was a '^matching" task*; four 
equations on one side of the page and five dot pictures on the other side. 
The crucial directions were, '^figure out which dot picture shows' (each) 
nunfcer sentence best... there are five pictures and only four nunfcer 
sentences. . .make up a nurfcer sentence that goes with the picture that*s 
left' .over." 



Example: 5-^>8 



Subtest 5: Conputation 
but different itemsT] 



(Same directions as Subtest 5 in booklet A 
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School A, Second Grade MANS Mean Scores 



t [ 

1 L 


1978 


1 

1979 


1980 


Mans Test A 










1 7 


1 • U 












A3: Functions (5 items) 

• VIII W W 1 ^/ 1 1 w jr w ■ W w 11 f / 


0.8 


1 .2 


1.3 


A4: Number Line Esti-mation (13 items) 


3.5 


3.5 


4.0 


AS* rnmniif^ f i nn T fl? ifprn^) 


4.4 


5.2 


4.9 


MANS Test B 








Bl: Labeling Number Lines (8 items) 


3.3 


3.8 


,4.1 


B2: Number Sentences 


1.9 


3.4 


2.9 


83: Word Problems (7 items) 


2.5 


3.1 


3.2 


84: Number Sentence Pictures (9 items) 


3". 2 


4.2 


" 3.9 


85: Computation II (12 items) 


. 4.1 


5.5 


~ 4.2 


Total MANS 


27.6 


3€.7 


34.5 



School B, Second Grade MANS Mean Scores 



Test 


1978 


1979 


1980 


Al: 


.Sequences (6 items) 


3.9 ^ 


4.7 


4.9 


A2: 


Equation Fluency 


3.9 


7.5 


6.9 


A3: 


Functions (5 items) 


3.0 


3.8 


.;.;3.5 


A4: 


Number Line Estimation (13 items) 


5.7 


7.7 


7.5 


A5: 


Computation I (12 items) 


7.9 


9.7 


9.8- 


Bl: 


Labelling Number Lines (8 items) 


6.1 


6.9 


7.1 


82: 


Number Sentences 


3.9 


6.7 


6.0 


83: 


Word Problems (7 items) 


5.2 


5.4 


5.4 


84: 


Number Sentence Pictures (9 items) 


5.5 


7.3 • 


6.9 


85: 


Computation U (12 items) 


'8.4 


9.1 


9.5 




Grand Total 


53.7 


68.8 


67.5 



<9 



2:, 

19 



Appendix B 
Third Grade MANS Scales and Schoo] Means 



1 



The means for each scale are given'for Schools A and B for 1978, 1979 and 
^ 1980 on page*s 24 and 2^. Then, on. page 26, th§ means for School A, fourth 
grade, are given. 



SCAL£ Al: H«i^t and Weight Table (6 items) 



ABSTRACT; 

SAMPLE:' 

SCALE A2 : 
ABSTRACT: 

SAMPLE: 

SCALE A3 : 
ABSTRACT: 



SCALE AA : 
ABSTRACT: 

SAMPLE: 



Read and interpret data from a table of students' weights and heights for 
two different years 

Who atayed the same height? 
Estimation (25 items) 

Quickly estimate which of 5 standard intervals contains the answer to each of 
a series of computation ijroblems. Three separate pages containing 8 addition, 
8 subtraction and ^7 multiplication problems respectively. 



100 - 9'5 

Functions (8 itemfl) 



10 



50 



too 



500 



1000 



For each of several problems, determine from 3 pairs of numbers what the 
"secret rule" is which produces the second Oumber from the first, and use it 



1 


CO find the missing 


SAMP^iE: 


tlm's Gams 



















Two Stage Word Problems (5 items) 

Word problems (printed in booklet ^and read by tester) in which two 
different operations must be performed and where the numbers in the 
given data are relatively small. 

Our h«M Iat ^ •tS* «^rT <i*T' 

C4ch 4<y mc ^ of thui mii ch« och«rt mvf . 

Durtns th« a«xt 5 d*Tt ho^ »«ttT tu* «*'»y' 



Isee CSMP Evaluation Report 4-B-2 for a fuller descri ption of these tests and 
. the results from the CSMP-non-CSMP'comparison involving &5 cUsses. 

21 ^'f , ^ 



» SCALE A5 : Large Number Computations (12 items) 

ABSTRACT: Solve computation problems given in an open sentence format, with the 
boxes sometimes in non-standard positions, and with numbers in the 
hundreds but relatively easy to work with (addition, subtraction and 
multiplication) . 



^,AMPL£: 



50p 



SCO 



SCALE A6 : Number Line , Labelling (8 items) 

« ■* ' ♦ 

ABSTRACT. Label the indicated "mark'* on several number lines, where marked intervals 

vary from item to item.aud viiAXft ochjar mark a axa jLr^aguJLarly Lab^llftd. 



SAMPLE: 



n 



IS 



19 



21 



SCALE A7: 



ABSTRACT: 



SAMPLE: 



Hints and Problems (5 items) 

Quickly complete a given addition problem by using the answer to another 
problem where one addend is the same as, and one is only slightly different 
from the given problem. 



Hint: 537 - 293 • 83 
537 * 283 Q 



SCALE Bl : 2 or 5 or 10. (10 items) 



ABSTRACT: 



SAMPLE: 



Quickly estiiiiate whether a given number is about 2 or 5 or 10 times 
as large as another given number. 



60 i3 about 



time's as lorge as 31 



SCALE B2 : Compoaite Functions (9 items) 



ABSTRACT: 



SAMPLE: 



ERIC 



Starting with a given number, apply one or »ore operations in sequence and 
determine final result;^. Also, same process except final result is known 
and star.ting number is to be determined- 





4 =^ Bi I I Mary 



2o 



22 



SCALE B3 : Equation Fluency 

ABSTRACT: Given the symbols: - -f - x 1 2 3 ( ). 

construct as many different equations as possible 



SAMPLE 
ANSWER: 



3-1 



SCALE BA : Circle the Larger (13 Items) 



ABSTRACT: 



Given pairs of computation problems, quickly determine which one has the 
larger answer, « ' 



SAMPLEt 



371 ^ 248 



370 + 258 



SCALE B5 : Missing Digits (8 items) 

ABSTRACT: Given a computation problem with one or two digits of the problem 

crossed out, determine whether or not the .given answer could have been 
right (before the digits were crossed out). 



SAMPLE: 



54 
511 

500 



Could 500 be the answer? 



No, yOQ it too maXI.O 
Tm, 300 CQuId b« rlfhcO 
Ho* 500 is coo blg.D 



SCALE B6 : Word Problems with "Raunding" (5 times) 



I 



ABST^A<!T: Solve word problems (printed in booklet and read by the tester) involving 
divlElon in which the given numbers do not divide evenly - i.e., the 
«Mirer, vhich aust be an integer, .can be obtained by rounding the obtained 
quotient up ef <fo%m. The numbers" of the given data ware relatively small ♦ 



SAMPLE: 



ABSTRACT: 



SAMPLE: 



An «ltv«tor c*n*t bold morm than 5 p«opl«. 

23 r«ofl« w«nt to Ti6m to th« tof floor. 

Host mm%y timm$ vUl th« tl«v«eor hav« to go up7 



_SCAL£ B7 : Fractions (8 Itemfl) 



Solve problems of the form x of y - I I 
1/2 or 1/3. ^ 



or X 



y where x Is 



y of 12 
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School A, Third Grade MANS Mean Scores 





Thi rd Grade 




1978 


1979 


1960 


.MANS Test A 








A): Hedght & Weight Table (6 items) 


2.2 


2.5 


2.4 


A2: EstNJmation (25 items) 


6.4 


7.8 


8.9 




1.1 


2.6 


2.2 




1.2 


2.0 


2.4 


A^' 1 arop Nrt romniitati on (1? itprisl 


- 2.9 


4.0 


2.8 


Afi* Niimbpr I inp La hp lino (8 itpms) 


^ 2.2 


3.4 


3.0 




0.8 


1.5 


1.3 


HANS Test B 








31: 2 or 5 or 10 (10 Items) 


4.5 


4.^ 


5.4 


B2: Composite Functions (9 items) 


3.1 


4.4 


5.r 


Dj. itquoLion r luenc)* 


3.5- 


5.2 


8.3 


84: Cirj:le the Larger (13 items) ' 


6.7 


7.4 


7.5 


B5: Missing Digits (8 items) 


2.7 


2.8 


2.8 


86: Word Problems Rounding 95 items)-. 


i.e 


1.2 


1.3 


87: ''Fractions (8 items) , - 


1.7 


. 2.9 


■3.1 


Total «ANS ' ' 


40.5 


52.7 


57.4 



i 




2i 

24 • 



School B, Third Grade MANS, Mean Scores 



1 



Test 


1978 


1979 


Al: 


Height and Weight Table (o items; 


5.1 ' 


4.8 


A2: 


Estimation (25 items; 


12.8 


15.0 


A3: 


Functions i. terns; 




6.4 


A4: 


Two Stage wora rroDiems \o iiems; 


3 8 


4.2 


A5: 


Large Number computation [ic iiems; 


7.4 


8.7 


Ao: 


Number Line Label nng [o iiems; 


6.1 


6:4 


A7: 


Hints and Problems (5 Items) 


3.0 


* 3.3 


Bl: 


2 or 5 or 10 (10 items) 


7.2 


8.0 


B2: 


Composite Functions (9 items) 


7.4 


■ 8.3 


83: 


Equation Fluency 


8.0 


7.4 


- B4: 


Circle the Larger (13 items) 


9.6 


10.5 


B5: 


Missing Digits (8 items) 


3.7 


4.3 


86: 


Word Problems Rounding (5 items) 


3.3 


3.6 


B7: 


Fractions (8 items) , 


■5.1 


6.5 




Grand Total 


88.3 


97.4 



# 



If. 




ERIC 
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School A, Third Grade MANS Tests 
Mean Scores for Fourth Graders 





Fourth Grade 




1978 


1979 


1980 


MANS Test A ^ 








AI • Hpiaht & Welaht Table (6 Items) 


2.8 


2.9 


2.9 


A7- Fstimation ( 25 items) 


8.4 


9.5 


10.4 


A^- Functions (8 Itetns) 


1\8 


2.7 


3.9 


A4 • Two Staop Uord Problems (5 items) 


1.8 


2.1 


2.6 


A^' 1 ;irnp Nn fomoutat'ion (12 1tems).v^ 


4.4 


4.6 


5.9 


AA> Niimhpr Mnp l^hplina-ffi items) 


3 1 


'^3.7 


4.0 


A7: Hints and Profalerns (5 items) 


1.2 


1.7 


1.9 


MANS Test B 








81: 2 or b or lU \\\J items; 


4.9 


5.2 


5.9- 


B2: Composite Functions (9 items) 


3.1 


5.9 


6.1 


83: Equation Fluency 


4.1 


5.6 


5.7 


84: Circle the Larger (13 Items) 


7.2 


7.6 


8.1 . 


85: Missing Digits (8 items.) 


3.0 


3.1 


3.0 


86: Word Problems Rounding 95 items) 


1.3 


2.4 


1.6- 


87: Fractions (8 items) 


1.8 
« 


3.5 



2.6 


Total MANS 


48.8 


60.9 


64.9 



it 



ERIC 
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Appendix C 

Fourth Grade MANS Scales and School B Means^ 

•The mean scores are given on page 32. 

Computation Seal es 
Scales 1-4: 

There were four computation scales, one in each of the basic operations, 
containing from 5 to 7 test items each. The format was mixed between horizontal 
and* vertical and the difficulty ranged from basic facts to algorithms (simple two 
digit multiplication and one-digit division). 

< 

Word Problems 

Scale #5, One-Step Word ProbTems . T^lese problems involved only one 
operation with a low level of computational difficulty. 




Scale #6, Two-Step Word ProbleTns . A fairly typical item of this scale 
was the following: 

There are 40 apples in my barrel now: 
I will eat 2 apples 'every day. 

How many apples will be left in my barrel after 5 days. 
This particular item required either repeated subtraction, or multipl ication and 
then subtraction. Other items required different combinations of operations. 

Scale #7, Hits and Misses . This scale concerned the calculation of game 
scores in which "hits" were worth 5 points while "misses*' lost 3 points. 

Scale Miscellaneous Word Problems . This scale contained very simple 
probTtfms involving concepts related to decimals, proportions and rounding. 

^See CSMP Evaluation Report 5-B-l for .a fuller description of these tests and ^ 
the results from the CSMP-non-CSMP comparison involving 17 classes. 
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" Large Number Computation 
Scales iS and 10. 

These two scales were composed of problems given In an open sentence, format 
illustrated by two examples below, | 1 x 4 « 400 

I 

800 Q « 200 . 

The probletns in Scale 9 contained either adclition or division signs and the 
problems in Scale 10 contained either subtraction or multiplication signs. Of course 
the process used by the student may or may not correspond to the written sign, as in 
the first example above |rtiere the student might ask either "4 times what equals 400?" , 
or ''Wh^t 1s 400 -r 4?" 

Jhe numbers were easy to work'with, usual ly multiples of one^ndred or fifty. 

Estimation 

Scales n4a) 14b) and 14c) each contained a series of computation problems 
using addition, multiplication and division respectively, and the task was to 
detennine which of several given intervals co'ntained the answer to the problem. ^ 
For example, which interval contains the answer to 2 x 49? 

(0.10) (10,50) (50.100) (100,500) (500,1000) 
One doesn't need to know that 2 x 49 = 98. merely tRat It's less than 2 x 50 = 100 

Scale #16, Multiplying by 2, 5 or 10 . posed several questions like the following: 

"100 is about times as large as 19" 

where the blank was to be filled in by whichever of 2, 5 or 10 was best, 

* ' Scale #17, Which is Larger , gave a series of pairs of computation problems and 
in each pair the student had to select the problem which had the largest answer, for 
example, 38 x 38 versus 39 x 31. Altogether there were 12 test items, 4^each dealing 
with ntiltipli cation, division and fractions. 



ERIC 



Number Relationships 



Scale #18, Solving H\M>er Machines , This used "inachirn?s''Jike the following: 




The left hand circle is always for the ninnber we started with and the right hand 
circle is for the number we ended up with after the machine had done its job; in th\s 
case subtracting 3. For each problem of Scale 18, an unlabeUed machine was shown in 
operation with various pairs of inputs and outputs, and then with one of the inputs 
or outputs missing. The task was to figure out from the given information what 
the machine was doing each time, and then to figure out'^the missing input or output. 

Scales #19 fusing Number Machines , also used machines, but they were always labelled 
and usually hooked up with other machines. For example: ^ 



The task was to figure out the missing input (In this case) or output. 



Scales #20a, 20b, 20c. These all dealt with "boxes"," as in the "squares" 
of a crossword puzzle. For any given problem, the numbers are always increasing or 
decreasing by the same fixed amount when one goes horizontally and by some dif- 
ferent amount when one goes vertically. -A typical item from Scale 20a) asked 
whether 86 would ever be in one of the boxes (which extend botb ways) of: 



36 



31 



26 



21 



3^ 



29 




A typical item from Scale 20b) asked the students to fill In the heavy boxes% 
the following diagram, having been told that horizontal counting Is by 6 and verti^ 
counting is by 5. 



25 



0 



37 



28 



33 



38 



40 



45 



50 



55 



Scale 20c,) also required completion of boxes except that instead of "counting by" 
there was "multiplying by". 



fm 



Other Scales 



The remaining 5 scales did not fit into one of the previously described groups 
of scales. 



1 ... 1 



#11 Fractions: ' Ther^ were two kinds of Items. Four Items required taking j °^ 3 
of a number; the other four items required addition df siib^traction of fr'acti on^ wi th 
like denominators. 



#12 Measuring: In each item of this scale, a rod was shown aligned in some way 
with a ruler (not always starting at the zero mark). The task was to deter^pe how ^ 
long the rod was. various rulers being marked In halves, quarters and tenths of an inch. 
For example: « " ' 
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#13 Using Hints : Three very large addition and multiplication proM-eins^were 
given., complete with answer. In each problem, one digit of one of the numbers in the 
problem was crossed out"and a new one written above. The task was to figure out what 
the ansv»er to the problem would now be, given this change. This was a matter of 
ma^king the appropriate adjustment in the original answer. Not enough time was'igiven 
to get the- new answer from scratch. 



#15 Labelling Number Lines; The items were of the following type: 

I . 4- ■ I J- ^ 1 
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%. #21 Place Value: In the items of this scale, a. number line marked in either 

'- 

li's. lOO's. or lOOO's was given and the task was to either indicate with an x on 
^ the number line approximately where a given number was located or name the approx- 
imate h.umber. located by a given x. For example: 




,5'"° Y~t^ «S ,7.000 



V 



3., 



E 



School B, ^ourth Grade MANS Mean Scores 



Test 


Mumber of 
■ Items 


m 1.978 


1979 


1980 


Computation 
#1 Addition 
#2 Subtraction" 
#3 HuUiplication 
#4 Division 


5 
5 
7 
fi 


4.5 
4.0 
5.3 
3.7 


4.5 
3.6 
5.9 
4.8 


4.7 
4.3 
6.3 
■ 5.1 


Word Problems 
#5 One-Step 
#6 Two-Step 
#7 Hits and Hisses 
#8 Hiscellaneous 


6- 
. 5 
6 


6 5 
3.7 

1.4 - 
2.5 


0 . 3 

4.5 - 

1.9 

3.6 


6.8 
4.9 
2.5 
4.2 


Number Relationships 
#18 Solving Nunter Machines 
#19 Using Number Hachines 
#20a) Boxes-1 
#20b) Boxes-2 
#20c) Boxes-3 


8 
9 

9- 

9 
7 


6.2 
5.6 

6.4 \ 
5.7 

- 2.9 


7.2 

7 ? 

— 7.5 
7.0 

4 Fx 


7.3 
7.2 
7.6 
7.0' 


Estimation 

#14a) Estimation-Addition 
#14b) Estimation-Multiplication 
^ #14c) Estimation-Subtraction 
#16 Multiplying by 2, 5 or 10 
#17 Which is Larcer 


8 
7 
7 

13 
12 


6.5 
5.0 
■ 3.8 
9.7 
8.7 


6.7 
5.8 
4.2 
9.9 
9.7 


^ S 

6.8 
5.8 
4.4 
10.9 

9.8 


Large Number Compuation 
#9 Large Number Computation {+,-^) 
fflu Larqe numDer Lomputation (-,x) 


9 

8 


5.4 
5.0 


6.5 
5.4 


7.9 

6J] 1 


Other Scales =-* , 
111 Fractions 
#12 Measuring 
^ #13 Using Hints 

#16 Number Line Labelling 
#21 Place Value 


8 
6 
5 

6 - 
8 - 


5.4 
4.0 
2.4 
3.3 
5.8 


6.4 
4.2 
2.7 
4.9 
6.0 


6.9 
4.4 
3.0 
5.2 
6.1 


Total - 




123.4 


141.7 


150.4 



r 
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